The Thing We Tried That Worked: Utile Distinctions for Relational Reinforcement Learning

This paper introduces a relational function approximation technique based on McCallum’s UTree algorithm (McCallum, 1995). We have extended the original approach to handle relational observations using an attribute graph of observable objects and relationships (McGovern et al., 2003). Furthermore, we address the inherent challenges that arise with a relational representation. We use stochastic sampling to manage the search space (Srinivasan, 1999), and sampling to address issues of autocorrelation (Jensen and Neville, 2002). We prevent the algorithm from growing an overly large and complex tree by incorporating Iterative Tree Induction’s approach (Utgoff, 1995). We compare Relational UTree’s performance with similar relational learning methods (Finney et al., 2002) (Driessens et al., 2001).